An unfortunate result of technological advances in image capture and reproduction is illegal copying and distribution of image content, in violation of copyright. One solution for counteracting illegal copying activity is the use of image watermarking as a forensic tool. Sophisticated watermarking techniques enable identifying information to be encoded within an image. A digital watermark can be embedded in the image beneath the threshold of visibility to a viewer, yet be detectable under image scanning and analysis. As just a few examples: U.S. Pat. No. 6,239,818 (Yoda), discloses embedding a pattern in a color print and adjusting cyan, magenta, yellow, black (CMYK) values such that the embedded data matches the color of the surround when viewed under a standard illuminant; commonly assigned U.S. Pat. No. 5,752,152 (Gasper et al.) discloses a pattern of microdots, less than 300 μm in diameter, for marking a photographic print that is subject to copyright.
Illegal copying is a particular concern to motion picture studios and distributors, representing a noticeable source of lost revenue. Watermarking of motion picture images would enable the source of an illegal copy to be tracked and would thus provide a deterrent to this activity. Watermarking techniques for still images and prints, however, may not be well-suited to motion picture film media. An encoded pattern that might not be easily visible within the single image of a print could become visible and annoying if it appears in a sequence of image frames. Moreover, a motion picture watermark must be detectable from a copy, such as a videotape copy, that is typically captured in a timing sequence that varies from the timing of motion picture frames through projection equipment and with varying image resolution, lighting, and filtering. For these and related reasons, motion picture watermarking requires a special set of techniques beyond those normally applied for still images.
A number of watermarking methods for motion images have been described in prior art patents and technical literature. Included are methods that apply a spatial-domain or frequency-domain watermark. In either approach, many techniques make use of a pseudo-random noise (PN) sequence in the watermark generation and extraction processes. The PN sequence serves as a carrier signal, which is modulated by the original message data, resulting in dispersed message data (that is, the watermark) that is distributed across a number of pixels in the image. A secret key (termed a “seed value”) is commonly used in generating the PN sequence, and knowledge of this key is required to extract the watermark and the associated original message data.
Among prior art patents that address watermarking methods for motion picture image content are U.S. Pat. No. 5,809,139 issued Sep. 15, 1998 to Girod et al. entitled “Watermarking Method and Apparatus for Compressed Digital Video”; U.S. Pat. No. 5,901,178 issued May 4, 1999 to Lee et al. entitled “Post-Compression Hidden Data Transport for Video”; and U.S. Pat. No. 5,991,426 issued Nov. 23, 1999 to Cox et al. entitled “Field-Based Watermark Insertion and Detection”. However, the methods disclosed in these patents can be applied only to a digital video data stream and are not directly applicable to motion picture film.
U.S. Pat. No. 6,026,193 issued Feb. 15, 2000 to Rhoads, entitled “Video Steganography”, discloses the basic concept of using multiple watermarked frames from an image sequence to extract the watermark with a higher degree of confidence than would be obtained with only a single frame. U.S. Pat. No. 6,449,379 to Rhoads entitled “Video steganography methods avoiding introduction of fixed pattern noise” proposes an improvement to this scheme by changing the PN carrier from frame to frame, for example.
Another approach to applying a watermark without the disadvantages of a fixed watermark pattern is to use a three-dimensional watermark pattern. An example of such a method can be found in a paper by J. Lubin et al, “Robust, content-dependent, high-fidelity watermark for tracking in digital cinema,” in Security and Watermarking of Multimedia Contents V, Proc. SPIE, Vol. 5020, Jan. 24, 2003. This paper discusses a method for embedding, into successive image frames, a watermark containing low frequency content in both the spatial and temporal dimensions. The method described by Lubin et al. may provide a temporally distributed watermark that is relatively robust. However, this method suffers from a key limitation for temporally distributed watermarking schemes: the requirement for temporal synchronization in order to recover or decode the watermark. That is, some method must be provided that allows indexing of each image frame with a reference frame; a sampling of successive image frames must include this reference in order to allow synchronization of watermarked frames and subsequent decoding. Significantly, the method described by Lubin et al. requires prior knowledge of the image content before application of a watermark is possible. Thus, this method would not be suitable for use as a pre-exposure scheme by a film manufacturer.
While a number of different approaches have been attempted for watermark application to motion pictures, there is considered to be room for improvement. Specifically, for motion picture film media that is watermarked using an exposure of a watermark pattern, there are limitations to these conventional approaches with respect to the color information of the watermark pattern itself. In relation to this color information, conventional approaches fail to consider one or more of the following problems:                (1) the inherent sensitivity of motion picture film media to different colors;        (2) the effect of a watermark exposure on the sensitometric response of the film; and,        (3) the color processing and associated distortions that can occur when a motion picture is illegally captured using a camcorder and subsequently distributed using compression techniques such as MPEG.        
In many watermarking techniques for color media, the watermark pattern is exposed using all three color planes (Red, Green, and Blue, referred to as RGB). Stated alternately, the watermark pattern is exposed onto all three colorants, such as dye layers (cyan, magenta, and yellow, referred to as CMY) for a photosensitive medium. This approach can provide a watermark with a neutral color that is substantially robust with respect to the various color distortions that can occur during illegal capture and distribution. However, while a three-color watermark exposure may work suitably for many types of color film and print media, there are problems specific to motion picture print films. In this class of film types, the respective photosensitive emulsions that are used to provide each of the three RGB color planes vary significantly in sensitivity. For most types of motion picture print film, the photosensitive emulsions for color printing that are sensitized to Green and Blue light are more sensitive to exposure energy than is the emulsion that is sensitized to Red light. Because of this, depending upon the writing technology that is employed to provide the watermark exposures, it may be difficult to achieve the necessary exposure levels for all three photosensitive emulsions. This problem is particularly pronounced for high-speed fabrication of motion picture print film.
As is well known in the imaging arts, the primary (additive) RGB colors are formed by imaging onto their complementary (subtractive) cyan, magenta, and yellow (CMY) colorant dye layers. Parts of the image that are not Red are imaged in the cyan dye layer. Parts of the image that are not Green are imaged in the magenta dye layer. Parts of an image that are not Blue are imaged in the yellow dye layer. Referring to the color sensitivity chart in FIG. 1, with sensitivity graphed on a log10 scale, the magenta and yellow colorant dye layer sensitivity curves for Blue and Green color planes show a marked increase in response to exposure energy over the cyan (Red-sensitized) curve. For some types of watermark application, the need for higher exposure levels for the Red color plane would not be a drawback. However, where speed is important, such as for pre-exposure of a watermark during film manufacture, for example, the low sensitivity of the cyan dye producing Red layer could slow the pre-exposure process or require high-energy exposure sources in the Red spectrum.
An additional problem relates to the impact of watermark application on image quality. The exposure of a conventional neutral watermark pattern onto a color photosensitive medium adds an overall density to each of the three RGB color planes. This effect changes the sensitometric response of the film to the actual scene content exposure and may even render image quality unsuitable, due to unwanted color shifts and tone scale distortion, unless appropriate corrections are made.
The density-to-log-exposure (D log E) graph of FIG. 4 compares sensitometric characteristics of one sensitized layer of a print film with and without a pre-exposed watermark. A curve 30a shows normal D log E response of an unexposed film layer. A curve 30b shows this response when a watermark pattern has been pre-exposed on the film layer. A third curve 30c shows the sensitometry adjustment needed to compensate for watermark exposure. This adjustment is carried out by changing emulsion response characteristics for the particular dye layer of the print film.
As disclosed in co-pending applications “Method and Apparatus for Watermarking Film” by Roddy et al., U.S. Ser. No. 10/364,488 and “Method Of Image Compensation For Watermarked Film” by Zolla et al., U.S. Ser. No. 10/742,167, cited above, a preferred approach to compensate for this problem is to reformulate the photosensitive emulsions, correcting for the watermark exposure and response, as shown in the example of FIG. 4, in order to provide the same effective response to image content exposure as if there were no watermark exposure. Using this approach, if a neutral watermark is produced by exposing all three color planes with a watermark pattern, it is then necessary to re-formulate all three photosensitive emulsions. It must be observed that emulsion reformulation is a difficult process, requiring careful process adjustments and testing, potentially adding considerable expense to the manufacturing process.
One solution that has been proposed for other types of color photosensitive medium is to apply a watermark only to a single color plane. This is the approach, for example, disclosed in U.S. Pat. No. 5,752,152 (Gasper et al.) where only Blue exposure is used for marking a photosensitive medium. Blue exposure results in a yellow watermark pattern, which is known to be less visible to a human observer than watermark patterns using other colors or a neutral color. However, while this method works well for its intended application, such a single-color watermark would not be particularly robust against the color processing and imaging distortions that are typically introduced during the illegal capture and distribution of motion pictures. The camcorder itself is often less sensitive to color in specific channels, due to an unequal distribution of Red, Green, and Blue sensing elements, as is described subsequently. Moreover, compression techniques such as MPEG use a luminance/chrominance color representation, discarding at least some portion of the chrominance information, because it is less perceptible to a human observer. Even if a different color plane is used, this single-channel method may not provide satisfactory results. Detection of a watermark pattern encoded in only a single color may be difficult, depending upon scene content. As a result, a single-color watermark exposure may not persist in a copy that is illegally made, thus rendering the watermark useless for the purpose of tracking stolen content.
Referring specifically to motion picture print film, another problem with watermark exposure in the Red color plane relates to the encoding of the audio signal on the film. A length of motion picture print film provides not only image content, but also provides accompanying audio soundtracks and synchronization information. Referring to FIG. 2, there is shown a small segment of 35 mm motion picture film having successive image frames 12 plus a number of tracks of encoded audio, and an interframe space 16, is positioned between successive image frames 12. An analog sound track 18 is printed between the side edge of frames 12 and perforations 14. A DTS (Digital Theater Systems) soundtrack 26 is encoded between frames 12 and analog sound track 18. A Dolby digital sound track 22 uses areas interspersed between perforations 14, repeated on both sides. Another digital sound track 24, conventionally the standard SDDS (Sony Dynamic Digital Sound) track is encoded along edges of print film 10. Digital sound tracks 22 and 24 are redundant, typically appearing on both sides of print film 10 as indicated by digital sound tracks 22′ and 24′. For considerations of watermark application, it is significant to observe that analog sound track 18 and digital sound tracks 22, 24, and 26 are encoded onto print film 10 using exposure to light, in much the same way as frames 12 are exposed. For this reason, any imperfection in imaging quality of print film 10 may also impact audio quality. Film grain, dust, surface imperfections, and other imaging anomalies not only degrade image quality, but may also have an impact on audio quality.
Due to the requirements of traditional sensing circuitry using vacuum tubes, the colorant dye layers of early color motion picture films were unable to provide sufficient density for accurately encoding the audio signal. To remedy this situation, special processing has been used so that metallic silver content along analog sound track 18 is not bleached from the film surface. This special processing step allows analog sound track 18 to have higher density to IR radiation than film dyes alone could provide. More modern improvements to analog sensing circuitry, retrofitted to a large number of early projection units, now allow the use of dye-only sound tracks. This results in cost savings, since the added procedures are no longer needed for restoring metallic silver compounds to the area of analog sound track 18 for these projectors. Instead of reading a highly dense, silver-bearing analog sound track 18 imprinted on the film, the newer solid-state detection circuitry reads analog sound encoding in the cyan dye layer that provides absorption of light in the Red region. This means, however, that there is heightened sensitivity to Red wavelengths, blocked most effectively by cyan dye in the audio track. Thus, any type of watermarking signal having density in the Red spectral region could have an adverse affect on the encoded audio signal of analog sound track 18.
A further complication, related to this problem with Red color content, is that there is no pre-determined orientation of frames and analog sound track 18 and DTS sound track 26 for unexposed film. As the film is shipped from the manufacturer, one orientation may be more likely than its opposite; however, either negative or print film may be rewound before being exposed. Therefore, once print film 10 is manufactured, it cannot be determined in which direction a negative film or print film 10 will actually be exposed. Thus, for 35 mm print film, for example, it is not certain at the time of manufacture whether analog sound track 18 and DTS sound track 26 run along the line of perforations 14 nearest one edge of print film 10 or the other. As is observable from the plan view of FIG. 2, frames 12 are skewed to one side of print film 10 relative to width W, rather than being centered, to accommodate audio sound track 18 and DTS sound track 26.
A practical watermark exposure scheme, particularly one that can be used for pre-exposure, must address the problems of uncertain placement of frames 12 relative to width W, which directly affects robustness and straightforward detection, and of the need for encoding analog and digital sound tracks 18, 22, 24, and 26.
For photosensitive media in general, it is known that a watermark encoding can be digitally added to the image frame at the time of printing. Currently, however, digital printing is much slower than conventional optical printing techniques. Thus, in a mass-production environment, it would be impractical to require an all-digital exposure system in order to apply a watermark to a motion picture print film.
Fortunately, it is possible to expose a watermark at different times during processing of the photosensitive medium. For example, as has been practiced and is described in U.S. Patent Application 2003/0012569 entitled “Pre-Exposure of Emulsion Media with a Steganographic Pattern” by Lowe et al., a latent monochromatic or polychromatic image can be exposed onto the “raw” photosensitive medium itself, at the time of manufacture. Then, when the medium is exposed to form the image, the image frame is effectively overlaid onto the watermark pattern. Such a method is also described in U.S. Pat. No. 6,438,231 entitled “Emulsion Film Media Employing Steganography” to Rhoads. The Rhoads '231 patent discloses this type of pre-exposure of the watermark onto the film emulsion within the frame area of negative film, for example.
It can be appreciated that watermark pre-exposure would have advantages for marking motion picture film at the time of manufacture or prior to exposure with image content. A length of motion picture film could be pre-exposed with unique identifying information, encoded in latent fashion, that could be used for forensic tracking of an illegal copy made from this same length of film.
Given these considerations, it can be seen that conventional approaches, such as simply applying a watermark pattern from one edge of film 10 to the other in all color planes, could yield unsatisfactory results, impairing image quality, degrading audio quality, complicating the coating emulsion design, adding cost, and compromising the robustness needed. At the same time, the watermark pattern for motion picture film media must have sufficient energy for detection in a copy of the projected film made using a camcorder device. Some improvement over conventional approaches is needed for providing watermark encoding that provides a good measure of robustness without introducing problems related to image and audio quality and that has minimal cost impact.